An Universal Image Attractiveness Ranking Framework

We propose a new framework to rank image attractiveness using a novel pairwise deep network trained with a large set of side-by-side multi-labeled image pairs from a web image index. The judges only provide relative ranking between two images without the need to directly assign an absolute score, or rate any predefined image attribute, thus making the rating more intuitive and accurate. We investigate a deep attractiveness rank net (DARN), a combination of deep convolutional neural network and rank net, to directly learn an attractiveness score mean and variance for each image and the underlying criteria the judges use to label each pair. The extension of this model (DARN-V2) is able to adapt to individual judge's personal preference. We also show the attractiveness of search results are significantly improved by using this attractiveness information in a real commercial search engine. We evaluate our model against other state-of-the-art models on our side-by-side web test data and another public aesthetic data set. With much less judgments (1M vs 50M), our model outperforms on side-by-side labeled data, and is comparable on data labeled by absolute score.Comment: Accepted by 2019 Winter Conference on Application of Computer Vision (WACV

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

We introduce Florence-2, a novel vision foundation model with a unified, prompt-based representation for a variety of computer vision and vision-language tasks. While existing large vision models excel in transfer learning, they struggle to perform a diversity of tasks with simple instructions, a capability that implies handling the complexity of various spatial hierarchy and semantic granularity. Florence-2 was designed to take text-prompt as task instructions and generate desirable results in text forms, whether it be captioning, object detection, grounding or segmentation. This multi-task learning setup demands large-scale, high-quality annotated data. To this end, we co-developed FLD-5B that consists of 5.4 billion comprehensive visual annotations on 126 million images, using an iterative strategy of automated image annotation and model refinement. We adopted a sequence-to-sequence structure to train Florence-2 to perform versatile and comprehensive vision tasks. Extensive evaluations on numerous tasks demonstrated Florence-2 to be a strong vision foundation model contender with unprecedented zero-shot and fine-tuning capabilities

Electronic Structure Models : Solution Theory, Linear Scaling Methods, and Stability Analysis

Various approaches have been developed to investigate materials in the last century. Quantum methods, solving the Schrodinger equation with different types of approximations, have best accuracy comparing with other methods. The most commonly used quantum methods are wavefunction method and density functional method. Wavefunction method is based on obtaining the wavefunction of the system. The simplest wavefunction method is Hartree -Fock (HF) theory. Density functional theory (DFT) considers electronic density as the basic variable instead of the wave function. Density functional theory offers a practical computational scheme, which is similar to Hartree-Fock method. Although Hartree-Fock and density functional theory have been studied many years, only a small number of papers discuss the mathematical properties of these models. We generalize the work of Suryanarayana et al. on local spin density approximation model, and developed finite-element formulations for Hartree-Fock and density functional theory models, including restricted Hartree-Fock, unrestricted Hartree-Fock, local spin density approximation, generalized gradient approximation, meta generalized gradient approximation and more generalized density functional theory models. We prove the well-posedness, and the existence of minimizers for these models in a finite calculation domain for both the all- electron problem and pseudopotential approximations. We also established the convergence of the finite-element approximation, and the convergence of finite element approximation with numerical quadratures by $\Gamma$- convergence for both the all-electron problem and pseudopotential approximations. It will be useful for the development of studying Hartree-Fock and density functional theory models numerically by finite element method. In Chapter 3, we present the mathematical proofs of the mathematical properties of these models. After well -defined models are proposed, numerical methods are needed to discretize and solve these problems. Much effort has been devoted to develop scalable real-space methods over the past decade , such as finite difference method, wavelet method, and finite element method. Difficulties arise when finite element method is used to solve a model with exact exchange energy, such as Hartree-Fock model. In order to handle exact exchange operator, we successfully proposed and numerically implemented a linear time cost and memory cost method with finite element bases. Multigrid method is used in our algorithm in order to achieve linear scaling. A variety of numerical experiments for atoms and molecules demonstrate reliable precision and speed. Our method could be generalized to density functional theory and hybrid models in a straightforward way. Each step in our self-consistent solver is well defined, and this makes the parallelization of our solver is feasible. In Chapter 4, we present the algorithms and numerical results of our linear scaling method. In Hartree -Fock and density functional theory solvers, molecular orbitals and orbital wavefunctions are solved by the Euler -Lagrange equation of the total energy functional. In order to establish whether that extremum is a maximizer, minimizer, or saddle point, the second functional derivative must be analyzed. We successfully proposed and numerically implemented a systematic way to study stability condition of Hartree-Fock and density functional theory models. We generated a few bifurcation figures for Hartree-Fock and density functional theory. Hessian analysis have also been conducted on extremums of Hartree- Fock and density functional theory models. A weakly bounded state is found when the nuclear charge of the two- electron atom gets really close to 1. It is a powerful tool to understand the stability of different solutions of Hartree-Fock and density functional theory models. In Chapter 5, we show a systematic way of stability study by Hessian matrix analysi